Search for: All records

Deep reinforcement learning (DRL) has proven capable of superhuman performance on many complex tasks. To achieve this success, DRL algorithms train a decision-making agent to select the actions that maximize some long-term performance measure. In many consequential real-world domains, however, optimal performance is not enough to justify an algorithm’s use—for example, sometimes a system’s robustness, stability, or safety must be rigorously ensured. Thus, methods for verifying DRL systems have emerged. These algorithms can guarantee a system’s properties over an infinite set of inputs, but the task is not trivial. DRL relies on deep neural networks (DNNs). DNNs are often referred to as “black boxes” because examining their respective structures does not elucidate their decision-making processes. Moreover, the sequential nature of the problems DRL is used to solve promotes significant scalability challenges. Finally, because DRL environments are often stochastic, verification methods must account for probabilistic behavior. To address these complications, a new subfield has emerged. In this survey, we establish the foundations of DRL and DRL verification, define a taxonomy for DRL verification methods, describe approaches for dealing with stochasticity, characterize considerations related to writing specifications, enumerate common testing tasks/environments, and detail opportunities for future research. 

This paper explores the feasibility of using sonification in delivering and communicating health and wellness status on personal devices. Ambient displays have proven to inform users of their health and wellness and help them to make healthier decisions, yet, little technology provides health assessments through sounds, which can be even more pervasive than visual displays. We developed a method to generate music from user preferences and evaluated it in a two-step user study. In the first step, we acquired general healthiness impressions from each user. In the second step, we generated customized melodies from music preferences in the first step to capture participants' perceived healthiness of those melodies. We deployed our surveys for 55 participants to complete on their own over 31 days. We analyzed the data to understand commonalities and differences in users' perceptions of music as an expression of health. Our findings show the existence of clear associations between perceived healthiness and different music features. We provide useful insights into how different musical features impact the perceived healthiness of music, how perceptions of healthiness vary between users, what trends exist between users' impressions, and what influences (or does not influence) a user's perception of healthiness in a melody. Overall, our results indicate validity in presenting health data through personalized music models. The findings can inform the design of behavior management applications on personal and ubiquitous devices. 

Academic achievement in the first year of college is critical for setting students on a pathway toward long-term academic and life success, yet little is known about the factors that shape early college academic achievement. Given the important role sleep plays in learning and memory, here we extend this work to evaluate whether nightly sleep duration predicts change in end-of-semester grade point average (GPA). First-year college students from three independent universities provided sleep actigraphy for a month early in their winter/spring academic term across five studies. Findings showed that greater early-term total nightly sleep duration predicted higher end-of-term GPA, an effect that persisted even after controlling for previous-term GPA and daytime sleep. Specifically, every additional hour of average nightly sleep duration early in the semester was associated with an 0.07 increase in end-of-term GPA. Sensitivity analyses using sleep thresholds also indicated that sleeping less than 6 h each night was a period where sleep shifted from helpful to harmful for end-of-term GPA, relative to previous-term GPA. Notably, predictive relationships with GPA were specific to total nightly sleep duration, and not other markers of sleep, such as the midpoint of a student’s nightly sleep window or bedtime timing variability. These findings across five studies establish nightly sleep duration as an important factor in academic success and highlight the potential value of testing early academic term total sleep time interventions during the formative first year of college. 

This paper presents a computational framework for modeling biobehavioral rhythms - the repeating cycles of physiological, psychological, social, and environmental events - from mobile and wearable data streams. The framework incorporates four main components: mobile data processing, rhythm discovery, rhythm modeling, and machine learning. We evaluate the framework with two case studies using datasets of smartphone, Fitbit, and OURA smart ring to evaluate the framework’s ability to (1) detect cyclic biobehavior, (2) model commonality and differences in rhythms of human participants in the sample datasets, and (3) predict their health and readiness status using models of biobehavioral rhythms. Our evaluation demonstrates the framework’s ability to generate new knowledge and findings through rigorous micro- and macro-level modeling of human rhythms from mobile and wearable data streams collected in the wild and using them to assess and predict different life and health outcomes. 

Feeling a sense of belonging is a central human motivation that has consequences for mental health and well-being, yet surprisingly little research has examined how belonging shapes mental health among young adults. In three data sets from two universities (exploratory study: N = 157; Confirmatory Study 1: N = 121; Confirmatory Study 2: n = 188 in winter term, n = 172 in spring term), we found that lower levels of daily-assessed feelings of belonging early and across the academic term predicted higher depressive symptoms at the end of the term. Furthermore, these relationships held when models controlled for baseline depressive symptoms, sense of social fit, and other social factors (loneliness and frequency of social interactions). These results highlight the relationship between feelings of belonging and depressive symptoms over and above other social factors. This work underscores the importance of daily-assessed feelings of belonging in predicting subsequent depressive symptoms and has implications for early detection and mental health interventions among young adults.